5 Literature-review Methods
5.1 What is a scholarly literature-review?
In a nutshell, doing a scholarly literature-review is like doing a sophisticated search of the internet. The word scholarly literature-review has two meanings, and both are important. One on the one hand, it can mean a review of scholarly literature. On the other hand, it can also mean a scholarly review of literature. In the former sense, scholarly describes the types of literature you’re reviewing. In the latter sense, scholarly describes your process for reviewing literature. A piece of literature is generally regarded as scholarly if it is produced by (and also typically for scholars) and it cites references that experts typically consider as valid evidence related to the author’s claims. A process for reviewing literature is loosely regarded as scholarly if the process is rigorous and well-organized. In these lecture notes, I will describe where you can find scholarly works, and how you can keep your search process rigorous and well-organized.
5.2 Why is it important to do a scholarly literature-review?
In sec-intro_types_of_scholarly_works, I briefly mentioned a few types of scholarly works that are typically produced by IWO psychologists. One of those is a literature-review study, which often culminates in a literature-review paper (for examples of literature-review papers, see: Legood et al., 2023; and Newman et al., 2017). However, on a smaller scale, arguably all scholarly projects involve at least some amount of literature-review. Broadly speaking, a literature-review is any process or result of searching for and discovering prior scholarly works related to your topic of inquiry. At a minimum, most published scholarly works provide their audience some amount of relevant literature-review that helps the audience understand the current study. Thus, even if you aren’t aiming to publish a literature-review paper, the process of conducting a literature-review is still important for your own education and for the education of your audience. This is true even if you want to conduct cutting-edge research on anything that is generally considered to be a revolutionary topic, because even such topics are contextualized and informed by the prior history related to such topics.
One of the main purposes for doing a literature-review is to avoid “reinventing the wheel”. At the risk of sounding overly reverent toward our elders, allow me to provide you an illustration. Decades before you were born, many scholars spent their entire careers thinking about many of the same questions you’re now wondering about. They wondered about the best ways to promote worker wellbeing, reduce stress, enhance performance, foster team cohesion, change company culture, etc., etc. Now imagine how much you’ve learned in the past 5 or 10 years of your life, and then imagine how much you’ll learn after a full career of researching IWO psychology topics! Those scholars, who came and went before you were born, published their thoughts and insights that they cultivated across their entire careers. Sure, some of it may be outdated in some ways, but many (arguably most) of the IWO psychology research from prior decades is still highly relevant today. Although human culture may seem to change rapidly, human nature changes very slowly at the pace of genetic changes in our collective DNA. Thus, by conducting a literature-review, we can reap in a matter of hours the knowledge of thousands of scholars who dedicated their careers to studying the same questions we’re interested in today. That is an extremely efficient way to learn a lot in a little bit of time.
Indeed, conducting a thorough literature-review has been and continues to be a feature of the praxiology of IWO psychologists. For better or worse, scholarly works produced by IWO psychologists have been criticized for overly lengthy literature-review sections that go into arguably too much detail about the past. Of course, there must be a balance. By knowing your target audience—what they likely know and don’t know—you can have a better idea of how much detail to include in the literature-review sections of your scholarly works.
Often, beginner researchers will lament, “I can’t research this topic because it’s already been done”. However, such thinking is sorely misguided! Even though other researchers examined phenomena related to your topic of inquiry, there is often so much more to be learned about that topic! Throughout the remainder of this course, I’m sure your eyes will be opened to all of the different types of questions that can be asked about any topic.
5.2.1 Types of scholarly works you can search for
In sec-intro_types_of_scholarly_works, I briefly mentioned a few types of scholarly works that are typically produced by IWO psychologists. Specifically, I mentioned: empirical studies, theory papers, lab studies, field studies, literature-review studies, and meta-analysis studies. Each of those types of studies typically culminates in one or more discrete products (also known as an output), which could be a written article, or a video, podcast, etc. In this chapter, I want to focus on written scholarly works, which are products that consist primarily of written language, because that is by far the most prevalent format available in scholarly databases (in subsequent lectures we will discuss other formats that heavily involve audio, video, and in-person events).
The very generic word that scholars use to refer to written works is paper (e.g. “I was reading a paper about…”, “I’m working on a paper about…”). Using more formal language, all written works start as manuscripts, which are unpublished written works. A manuscript might be unfinished, or it might be completely written. As long as it’s unpublished, we can call it a manuscript. When a manuscript becomes published, we call it a publication, which is any published work (in this case, a published written work).
Among scholars, almost any publication can be categorized as either a book or an article.
I’m sure you’re already familiar with books, but scholars distinguish between two important subcategories: authored books versus edited books. An authored book is written entirely or almost entirely by a single author or a small team of authors, such that the entire team gets credit for writing the entire book. In an edited book, one or more persons known as the editors of that book will create a plan for what they want the book to be about, and then they will invite several other authors—who are considered experts in the various topics related to the book—to write one or more specific chapters for the book. Typically, the editors are experienced scholars who are knowledgeable about the topics in the book, and they provide editorial feedback to the chapter authors’ chapter manuscripts for the book before a final version is reached. Each chapter may be written by one or more authors, and each author typically only contributes one chapter (though multiple chapter contributions isn’t unheard of).
In modern edited books, the book will be assigned a DOI, and each individual chapter will also be assigned a separate DOI. That way, the authors of each chapter can keep track of how many people are reading/citing/downloading their specific chapters. See
In the context of published written works, you may have some sense of what we mean by articles (e.g., “I was reading an article about…”). Among scholars, the word article almost always means a peer-reviewed journal article, which is an article that was reviewed by other scholars—who are considered experts in the topics of the article—before it is published in a scholarly journal. A scholarly journal is essentially a magazine for scholarly articles (and recall earlier in this chapter I mentioned that any written work is generally regarded as scholarly if it is produced by and for scholars and it cites references that experts typically consider as valid evidence related to the author’s claims). Before the internet, journals were published in print, and that’s why each new “print” from a journal is referenced via its volume and issue number (just like magazines). The volume typically corresponds to the number of years the journal has existed up to that point, and the issue is the number of times a new instance of the journal has been published that year. Modern journals are increasingly getting rid of issue numbers, because journals are almost exclusively accessed online without the need for any physical prints.
In sec-Disseminating, I provide more detail about the peer-review process.
In this modern era in which scholarly works are almost always produced digitally, you can think of any publication as a type of digital object. To help keep track of digital objects, technicians invented the Digital Object Identifier, or DOI (pronounced: /ˌdiː.oʊˈaɪ/, like dee-oh-eye) which is a globally unique alphanumeric identifier for a specified digital object (e.g., an article, dataset, picture, audio file, book, book chapter, preprint). They are convenient because they are persistent, meaning the DOI for an object won’t change, even if the website URL for the object is updated. That way, searching for and finding a digital object via its DOI is super simple.
5.2.2 Where to find scholarly works
An experienced researcher can find high-quality scholarly works from a simple internet search (e.g., Google search), because such researchers are skilled at figuring out whether any research article is rigorous and transparent in its methods. However, less-experienced researchers might not know how to differentiate a high-quality scholarly peer-reviewed journal article, versus an article that looks like a peer-reviewed journal article but is actually published in an outlet that enforces very low-quality standards for the scientific validity of the contents of its articles. To help you avoid getting misguided by poor-quality research articles, you can play it safe by sticking to publication outlets that have a long reputation of enforcing high standards for their content.
5.2.2.1 A word of caution about Google Scholar
In a moment, I will give you a brief list of good sources to get you started. But first, let me say something about Google Scholar. Google Scholar has done a lot of good, especially by simplifying access to many scholarly works that would otherwise be difficult to find. However, less-experienced researchers can be easily duped on Google Scholar, because it contains anything and everything that is even remotely classified as scholarly. In other words, it’s very easy to find low-quality research on Google Scholar. Therefore, unless you already have a list of high-quality journals you want to search for on Google Scholar, I would advise caution before you assume any papers on Google Scholar are high-quality. With time, you will learn how to identify a good paper (e.g., by evaluating the contents of the paper, the history of its authors, the history of the editors of the journal that published the paper, etc.), and then you can search Google Scholar with confidence.
5.2.2.2 APA Databases
If I had to recommend one database that is essentially guaranteed to only include high-quality publication outlets, it would be the APA PsycArticles® database—for which most universities pay a subscription on behalf of their students. The APA PsycArticles® database contains the scholarly peer-reviewed journals that are published by the APA Journals™ publisher, as well as other high-quality journals that are affiliated with the APA.
You may have also heard of the APA PsycInfo® database, which would be my second recommended database for new researchers—maybe even tied in first place with APA PsycArticles®. The APA PsycInfo® database contains many more journals than the APA PsycArticles® database (most of which aren’t affiliated with APA at all), but APA PsycInfo® doesn’t necessarily include the full text of the articles for you to read (often, it only includes the short summary paragraph known as the abstract), unless you have a separate subscription to those third-party providers for those articles. In contrast, APA PsycArticles® provides you with the full text for all articles included in the database.
Now might be a good time to explain to you the difference between a database, versus a host and/or platform. A database is a collection of data, and databases are typically updated to include the newest data into it. In contrast, a host and/or platform is merely a service that gives you access to a database. APA built a website interface—which it calls APA PsycNet®—as a platform where you can access all of APA’s various databases, including APA PsycArticles® and APA PsycInfo®. Similarly, you may have also heard of something known as EBSCOhost, which is another online platform for accessing many databases, including APA’s databases, depending on which databases you are subscribed to via EBSCOhost. Therefore, when describing where we will search for scholarly sources, we mustn’t say “I will search EBSCOhost”, because the databases you are subscribed to in EBSCOhost (perhaps via your university) may be very different than the ones I’m subscribed to, even though we’re both using the EBSCOhost platform. Instead, we must name the actual databases we are searching.
5.2.2.3 Web of Science™
Another good database for you to be aware of is the Web of Science Core Collection™, which is typically accessed via the Web of Science™ platform. Many researchers often refer to the Web of Science Core Collection™ as simply the Web of Science database—often abbreviated as WoS—but that is technically incorrect and confusing because the Web of Science™ platform can also be used to access other databases besides the Web of Science Core Collection™. Either way, the Web of Science Core Collection™ is a large database, and it does a much better job of filtering out low-quality outlets than Google Scholar does.
5.2.2.4 Library websites
Another resource worth keeping in mind are university library websites. If you are a student or employee of a university, your tuition dollars typically automatically help pay for dozens of database subscriptions which you can access via your library’s website. Depending on where you live, public libraries might also provide access at no charge to you. Most universities subscribe to APA PsycArticles® and APA PsycInfo®, and you would typically access them via a platform like EBSCOhost.
5.2.2.5 Journals that IWO psychologists frequently publish in
In addition to databases which contain written works across hundreds of journals, you can also browse the websites of specific journals. As you become more experienced, you will develop a sense of which journals typically produce high-quality content. There are hundreds of journals that IWO psychologists typically publish in.
Below, I’ve included a small list of journals that may be particularly good for people who are new to IWO psychology, because I think that will give you a concrete launching point. However, by presenting you a small list of examples here, I’m risking biasing you into thinking these are somehow better than any journals I didn’t include in the list. There are dozens of high-quality journals I omitted from this list, simply because I didn’t want the list to be overwhelming for you. If you’re ever wondering what other journals might be good to look into, a quick internet search should be helpful, and you can also email me such questions.
Personally, I recommend keeping a spreadsheet of your favorite journals (including their website URLS, and other information you find useful), and then making a routine for reading through the abstracts of those journals so that you can stay at least somewhat up-to-date on the latest research in your areas of interest.
- Journal of Applied Psychology (JAP)
- Personnel Psychology (P. Psych)
- Annual Review of Organizational Psychology and Organizational Behavior
- Journal of Occupational and Organizational Psychology (JOOP)
- Journal of Business and Psychology (JBP)
- Journal of Occupational Health Psychology (JOHP)
- Academy of Management Annals
- Academy of Management Review (AMR)
- Harvard Business Review (HBR)
- Human Resource Management Review
- Academy of Management Journal (AMJ)
- American Psychologist
- Psychological Bulletin
5.3 Now that we know where to search, what should we type into the search box?
The text that you write into any search box (e.g., on a website or computer database) is known as your query. Typically, the search boxes for scholarly databases are built to handle Boolean queries, which are queries that use Boolean logic. Boolean queries contain special symbols that enable you to be very specific with your queries. Table tbl-boolean_query_syntax contains the most important symbols you should know.
Boolean logic is named after George Boole (1815–1864) who introduced its foundational concepts.
| Syntax element | Example Boolean query | What it does in a query |
|---|---|---|
AND |
work AND "psychological safety" |
We are requiring both expressions that are on either side of the AND operator. In this example, we’re saying we don’t want any results that only contain work nor only psychological safety; we only want results that contain both expressions. |
OR |
occupation OR work |
We are requiring at least one of the expressions on either side of the OR operator. In this example, we’re saying we want any results that contain either occupation, or work, or both occupation and work at the same time. |
NOT (sometimes also the minus sign: –) |
work NOT physics NOT joule |
Anything after the NOT operator will be excluded from the results. In this case, we’re searching for anything that contains work, but we don’t want anything that’s about physics, which has a completely different definition of work. |
* (it’s an asterisk, but in queries it’s typically known as the wildcard symbol) |
work* |
We will get results containing any word beginning with work, such as worker, working, works, workable, etc. Some search engines allow wildcards inside a double quote, but some don’t. You can check the specifics of the search engine you’re using. |
" " (i.e., double quotes) |
occupational NOT "occupational therapy" |
Treat the contents inside the double quotes as a single intact phrase (also known as a string). In this example, we could end up with results that contain therapy, but not any results that contain occupational therapy as a sequence of two words. In other words, we could get results containing text such as occupation-related therapy, and therapy for occupations, but not anything that specifically contains occupational therapy. |
( ) (i.e., parentheses) |
(work OR organization*) AND (climate OR culture) |
Parentheses will group expressions in the same way you would for regular algebra (from grade school). In this example, we want results that are about work climate or work culture, which are also known as organizational climateor organizational culture. |
The other crucial thing to know is the types of database fields you can search within a search engine or database. Perhaps the best way to explain the concept of database fields is via an example. If you think of a spreadsheet, a field could be a column that you’ve labeled for a specific type of data. For example, if I want to create a database which will be a simple spreadsheet to list everyone in my company, such that each row of the spreadsheet will be a new person, then I might create these fields (or columns) in my spreadsheet: First name, Last name, Title, Department, Email. To make this example more concrete, see Table tbl-simple_database_example.
| First name | Last name | Title | Department | |
|---|---|---|---|---|
| Jane | Smith | Project Manager | Human Resources | janesmith@example.com |
| Alex | Xela | Engineer | Computer Science | alexxela@example.com |
| Ricky | Bobby | Marketer | Marketing | rickkybobby@example.com |
Unfortunately, scholarly databases don’t all use the same names for the same types of fields. For example, for a field that contains the title of a journal, one database might call that field Journal whereas another database might call it Outlet. However, most databases typically make it very easy for you to see a complete list of all the fields you can search, as well as how to format your queries to search those fields.
Thus, equipped with the knowledge of Boolean query syntax and the database-specific formats for how to search in fields, you can combine those two to make highly-specific queries that specify what you want to search in various fields. For example, you could specify you want to search for psychological* AND safe* in the Title field, and also Review in the Journal field. That way, you’d get results from any journal with the word Review in the journal’s title (such journals typically publish literature reviews authored by experienced scholars), and your results would contain things like psychological safety or psychologically safe in the Title of the article.
5.4 Specifying your inclusion and/or exclusion criteria
Now that we’ve discussed the types of scholarly works you could search and how to format your queries, I want to now focus on the important step of creating inclusion and/or exclusion criteria. A criterion is “a principle or standard by which something may be judged or decided” (Oxford Dictionary of English, 2025). Your inclusion criteria are your criteria that define the types of scholarly works you want to include in your literature-review. Your exclusion criteria are your criteria that define the types of scholarly works you want to exclude from your literature-review.
The purposes of inclusion/exclusion criteria are primarily to help you organize your literature-review process. Often, a peer-reviewer will ask you how why you decided to review only the scholarly works you ended up including in your literature-review write-up, or why you didn’t include other specific scholarly works that the peer-reviewer believes would be good to include. In that situation, having a clearly specified and logical list of inclusion/exclusion criteria will help validate your argument in response to that peer-reviewer. Indeed, if your inclusion/exclusion criteria are nonexistent or vague or illogical, the peer-reviewer might be correct in stating that your literature-review is severely incomplete due to major omissions of various types of scholarly works that you really should’ve included in the first place.
Importantly, your inclusion/exclusion criteria should be compatible with your goals for your research (e.g., your hypotheses). For example, if you want to figure out the average job-satisfaction of workers in the United States since the Great Recession of 2007, then your inclusion/exclusion criteria might look like this:
- I only want information that was published in high-quality outlets, such as high-quality journals, or official U.S. government databases (because I don’t want any results that came from a sloppy measurement of job-satisfaction, nor incorrect methods for calculating the statistics).
- I don’t want any conference posters; due to the high variability in the quality of peer-reviews for conferences.
- I want the information to be based on data that was collected after 2007 (if the date of data-collection is unknown, I’ll exclude those sources).
- I want information about working adults.
The above list is just an example. You could create a much more specific set of criteria. You might even write a list of specific outlets and justify the list on the basis of their quality and your limited time. The appropriate level of detail will depend on your preference and the preference of whichever outlets you’re aiming to publish in.
5.5 Specifying your list of keywords
When you’re searching in a typical search engine or database, you likely aren’t writing complete sentences. Instead, you’re probably only writing a fragmented series of keywords, like: BBQ restaurants or eyeliner tutorial how to.
Especially in scholarly literature reviews, the quality of your keywords can make a huge difference in the types of information you end up finding. This is especially true when scholars use different words to refer to one concept. For example, there is a concept in statistics known as Simpson’s paradox, but it is also known as Simpson’s reversal, the Yule–Simpson effect, and the amalgamation paradox. Imagine if you only searched for scholarly works that included the keyword Simpson's paradox, and you ended up not finding several newer publications that use the more recent keywords such as Yule–Simpson effect or amalgamation paradox.
Thus, it is extremely important to keep a list of all of the keywords that are relevant to your query.
Fortunately, recent developments in large-language artificial-intelligence tools such as ChatGPT are immensely helpful for coming up with a great list of keywords. For example, you could just ask ChatGPT something like, “Please give me a comprehensive list of keywords I should include in my queries for scholarly databases, to help me find works about…[insert your main ideas here]” (in sec-AI_for_litreview, I describe other ways for using large-language A.I. tools to help with literature-reviews).
In the past—before tools like ChatGPT—scholars relied heavily on thesauruses such as the APA Thesaurus of Psychological Index Terms® (which requires a paid subscription) and the APA Dictionary of Psychology (available for free online). Those resources are still very useful today, but tools like ChatGPT have made the keyword-creation process much easier. In fact, you could try asking ChatGPT to use the APA Thesaurus and APA Dictionary when it creates a keyword list.
5.6 Forward citation search and Backward citation search
Aside from entering queries into a searchbox, another excellent set of techniques to discover scholarly-works relevant to your query is the forward citation search and backward citation search.
When you are conducting a backward citation search, you are essentially asking “What came before this publication?”. In other words, you’re moving backward across time. Specifically, you are looking at the references that are listed in the References section of any good scholarly publication. For example, if I found a great journal article published in October 2020, I can look in the References section of that article and look up each of the sources listed in there. Logically, all of the sources in the References section were published before October 2020. Thus, we are searching backward across time.
When you are conducting a forward citation search, you are essentially asking “What came after this publication?”. In other words, you’re moving forward across time. Specifically, you are looking for any subsequent publications that are citing a specified prior publication. For example, if I found a great journal article published in January 1991, I can search databases to find any subsequent publications that reference the January 1991 article. Logically, any publication that references the 1991 article must have been published after January 1991. Thus, we are searching forward across time.
To do a forward citation search, you’ll need to use a search platform that supports that feature, such as Google Scholar (free), or the Scite® platform (requires a paid subscription), or the APA PsycNet® platform (which requires a paid subscription). It is particularly easy in Google Scholar, because all you have to do is click on the Cited by button beneath any search result. Keep in mind, the publications that will be listed in one platform may be slightly different than those of another platform, because of their different underlying databases.
Another tool that can help with both backward and forward citation searches is known as the Local Citation Network (free). To use it, you’ll just need to enter the DOI or some other identifier for your focal publication.
5.7 Using A.I. tools for literature-reviews
Around the year 2023, A.I. tools like ChatGPT exploded in popularity. Since then, those tools have evolved at such a rapid pace that it would be a bit of a waste of time for me to go into lengthy detail about the various features of such tools—because they are very likely to become much more advanced tomorrow! Thus, it seems email, phone, and in-person conversations are the better communication mediums for discussing current A.I. tools, rather than trying to convey those tools in these lecture notes which only get updated once per year.
Still, I here is a general list of use-cases (i.e., contexts or purposes for which something is used) for using tools like ChatGPT for literature-reviews:
- Summarizing a topic, ideally with cited references included for you.
- Getting help to re-specify your research topic more clearly.
- Creating an appropriate set of inclusion/exclusion criteria for the types of scholarly works you want to find.
- Creating a comprehensive list of keywords (including synonyms and near-synonyms) for your queries.
- Discovering publications that fit your inclusion/exclusion criteria and your keywords.
- Summarizing any publications that you’re struggling to understand or don’t have the time to read completely for now.
Elicit (https://elicit.com) is another popular A.I. tools specifically made for reviewing scholarly literature. There are likely many other tools, so look around!
5.8 Use reference-management software to organize your collection of scholarly-works
As you are conducting your literature-review, you will start to build a collection of good publications that are relevant to your topic. Many people will immediately download the PDF of each of those publications and sick them in a folder on their computer. That is certainly fine, but there is a much more organized and user-friendly method for managing your collection. I’m talking about reference-management software.
Reference-management software (also known as citation-management software, or citation managers) help you collect, store, organize, and cite digital objects that you may want to reference (e.g., scholarly works such as articles, books, chapters, conference posters, etc.). Typically, reference-management software also enables you to effortless and automatically create a perfectly formatted (e.g., APA style) Reference list from all of the sources you’ve included in your manuscript.
Reference-management softwares rely on meta-data for the digital objects in your collection. Of course, digital objects convey information encoded in that object—such as the words and other information contained in a journal article—but the digital object is also associated with meta-data, which is any data that describes the digital object itself. Examples of meta-data include:
- The names of the authors or other creators.
- The date the object was published or created (e.g., publication date, or conference date).
- The title of the object (e.g., book title, article title).
- The object’s DOI.
- The name of the outlet in which the object is published (e.g., journal title, or website name).
Zotero® is one of the most popular reference-management softwares, probably because of its robust features and it is free! Zotero®, like many reference-management softwares, can automatically import the meta-data for any digital object, based on the object’s DOI or URL. In fact, Zotero® has a web-browser extension that will let you import digital objects into your collection by just clicking a button when you’re on the website of that digital object.
At the beginning of this course I included a tutorial on how to set up and use Zotero®. I also enjoy talking about Zotero® in general, so please feel free to email me if you have any questions.